Composition of Feature Space and State Space Dynamics Models for Model-based Reinforcement Learning

Learning a dynamics model and a reward model during reinforcement learning is a useful way, since the agent can also update its value function by using the models. In this paper, we propose a general dynamics model that is a composition of the feature space dynamics model and the state space dynamics model. This way enables to obtain a good generalization from a small number of samples because of the linearity of the state space dynamics, while it does not lose the accuracy. We demonstrate the simulation comparison of some dynamics models used together with a Dyna algorithm.